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Abstract 

In the two Higgs doublet model type III and in several other extensions of the Standard 
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LFV coupling parameter A rM are presented. 
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In the two Higgs doublet model type III and in several other extensions of the Standard Model, 
there are no discrete symmetries that suppress flavor changing couplings at tree level. The ex- 
perimental observation of the v^-v-r flavor oscillation may suggest the non-conservation of lepton 
number. This would lead to the decay of the type A°/H° —* t*/^ . We determine the present 
low energy limit on lepton flavor violating (LFV) couplings from the muon g — 2 measurement and 
discuss the prospects for detecting lepton flavor violating decays at the TeVatron and at the Large 
Hadron Collider. The achievable bounds on the LFV coupling parameter A TM are presented. 
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I. MOTIVATION 

In the Standard Model (SM) , lepton flavor is conserved 
separately for each generation. The diagonalization of 
the up-type and down-type mass matrices ensures the di- 
agonalization of the Higgs- fermion coupling matrices |Q : 
the interaction term of the neutral fields in the SM can 
be written as: 



-hijipii/jjcf). 



(1) 



The spontaneous electro-weak symmetry breaking gives 
the mass matrix 



Mij = hij < 



> 



(2) 



Diagonalizing My also diagonalizes the Yukawa coupling 
matrix h^. The severe experimental limits on the ex- 
istence of flavor changing neutral currents place strin- 
gent constraints on the flavor changing sector of extended 
models |Q where lepton flavor violation (LFV) may ap- 
pear at tree level or may be induced at higher orders. In 
the Minimal Supersymmetric Standard Model (MSSM) 
the flavor problem is related to the soft supersymmetry 
breaking mass terms. In the basis where the lepton mass 
matrix is diagonalized, if there are non-zero off-diagonal 
matrix elements in the slepton mass matrix, LFV is in- 
troduced via loop contributions involving slepton mixing. 
There are many ways to avoid LFV, for example gravity 
H or gauge mediated Q supersymmetry breaking, or fla- 
vor symmetries Q. In the minimal super-gravity model 
(SUGRA) the supersymmetry breaking mass terms have 
a universal structure at a high scale of the order of the 
Planck scale. However LFV effects can be induced by 
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radiative corrections ||. Large LFV effects can arise in 
supersymmetric models (SUSY) with a right-handed Ma- 
jorana neutrino H |, g, |l| and in SUSY with R-parity 
violation ]Tl| . 

In general, in models with several Higgs doublets, the 
up-type quarks and the down-type quarks can simulta- 
neously couple to more than a single scalar doublet. As 
a result, the same operators do not diagonalize the mass 
matrices and the Higgs- fermion couplings, leading to the 
prediction of Flavor Changing Neutral Current (FCNC) 
at tree level. For instance in the two-Higgs Doublet 
Model (2HDM), the Yukawa interaction Lagrangian (for 
the neutral fields) can be written as: 



(3) 



which gives, after spontaneous electro-weak symmetry 
breaking, a mass matrix of the form: 



M, 



hi < W\ > +9ij < ¥2 > 



(4) 



When this matrix My is diagonalized, the coupling ma- 
trices fij and gtj are not, in general, diagonalized. To 
suppress tree level FCNC in the theory so as not to be 
in conflict with known experimental limits, an ad hoc 
discrete symmetry is invoked fl2|| whereby the fermions 
of a given electric charge could couple to no more than 
one Higgs doublet. In the 2HDM, the up-type and the 
down-type quarks couple either to the same Higgs dou- 
blet (this is known as the 2HDM-I) , or they could couple 
to different doublets (2HDM-II). One of the most strin- 
gent test of the 2HDM type I and type II comes from the 
measurement of the b — > sj decay rate which receives 
substantial enhancement (over the SM prediction) in the 
2HDM in a large region of the (rnu± , tan (3) parameter 
space fL3|, |l4j , fl5| . The measured b — ► 57 decay rate from 
CLEO |Jand ALEPH leads to a model dependent 
indirect lower bound of the charged Higgs mass as func- 
tion of tan/3 |f]. 

In the 2HDM-III, no discrete symmetries are present 
and in general FCNC exist in this model |l9], [2(J. As 



2 



an example the LFV interaction Lagrangian of the light 
neutral Higgs boson h of the 2HDM-III type b (see the 
Appendix for details) is: 



C 



LFV 



— hij liljh + h.c. = £j 



cos(a - 0) T , , 

v^cos/? 

(5) 

where a is the mixing angle of the neutral Higgs sector 
and £ij the Yukawa LFV couplings and i, j are the gen- 
eration indices (in the following the notation hij will be 
used to indicate the generic Yukawa coupling including 
the mixing angles). To be consistent with experimental 
data on K°-K°, D°-D° and B°~B° mixing which put 
stringent constraints on flavor changing couplings with 
the first generation index, and since one might expect the 
biggest contribution to come from the LFV couplings of 
the second and the third generation (£ s b, £ T/J , £ ct ), these 
couplings have been parameterized as a function of the 
masses of the fermions involved since a natural hierarchy 
is found in the fermion masses : 



(6) 



where v ~ 246 GeV and the residual arbitrariness of fla- 
vor changing couplings is expressed by the parameters Ay 
which is constrained by experimental bounds on FCNC 
and LFV processes. A similar hierarchy will be assumed 
for the LFV coupling raj of 2HDM-III type a — see the 
appendix. In the charged Higgs decays this implies a 
zero LFV coupling if the neutrino is massless and in gen- 
eral a suppression proportional to the square root of the 
small neutrino mass. We also consider an alternative 
case in which we drop the neutrino mass dependence for 
the charged Higgs LFV couplings and adopt instead the 
same parameterization as in the neutral Higgs sector. In 
the numerical analysis of the muon anomalous magnetic 
moment, this distinction is not important as the charged 
Higgs contribution is small in both cases and can be ne- 
glected in comparison to the neutral ones for the range 
of masses and mixing angles considered. 



II. LOW ENERGY BOUNDS 

In the purely leptonic sector, the fx — > ej conversion 
process gives y/\ Tfl \e T < 5 . It would be desirable to 
examine a process that depends only on a single coupling. 
One such process is the muon anomalous magnetic mo- 
ment a M = (<? M — 2)/2 [^] where high precision data ]23] ] 
can be used to constrain A rM , by comparing the measured 



to the theoretical prediction of the SM. The new ex- 



perimental world average reads 122 



cxp _ 



11659203(8) x 10 



-10 



(7) 



Recently, the standard model calculation was revised in 
order to take into account the correct sign for the light by 
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FIG. 1: The one- loop contributions of the Higgs sector to a M . 
H stands for a generic neutral Higgs boson, / is a lepton. 
With / = /i we obtain the flavor conserving contribution, with 
f — t the LFV one. As explained in the text we neglect the 
LFV contribution with / = e. 



light hadronic contribution 
expectation is : 



and the standard model 



,SM 



11659177(7) x 10 



Another often quoted value is 



,SM 



11659186(8) x 10 



-10 



-10 



(8) 



(9) 



which gives only slightly more restrictive figures if used 
to bound the LFV couplings. In the following we shall 
use the value of Equation (||) . Note that a recent evalua- 
tion of the light by light hadronic correction |2j| based on 
chiral perturbation theory suggests that the theoretical 
error due to unknown low energy constants from sub- 
leading contributions may increase the estimated error. 
The difference between experiment and the SM theoret- 
ical calculation is: 



Aa„ 



cxp 



SM 



"n -«„ - 26(11) x 10- 1U . (10) 

We obtain the 90% confidence level (CL) range on Aa M 



8 x 10" 



< Aa M < 44 x 10" 



(11) 



to constrain new physics. In the following we shall con- 
sider the effect of flavor violating Higgs-leptons interac- 
tions plus the flavor conserving Higgs bosons contribu- 
tions as the only additional ones with respect to the SM. 
At one-loop level the Feynman diagrams are those of Fig- 
ure |] and the contribution to is given for a large class 
of models by p6[ (see also the erratum in Q concerning 
other results in the literature), and can be used to ob- 
tain the one-loop Higgs contributions to a M for the model 
considered in this paper: 



Aa,, = 



h lf m l 
8tt 2 



x 2 (l — x) ± x 2 (rrif /m^) 
m,,x A 



x(m 2 



f 



r 5- dX 

(1 — x)m H 

(12) 

for a neutral Higgs boson and the sign is +(— ) for the 
scalar (pseudo-scalar), hy is here a generic Yukawa cou- 
pling, whose expression in terms of rjij or £y and the 
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angles a, (3 can be read in the Lagrangian given in the 
appendix, my is the mass of the muon for the flavor con- 
serving contribution with coupling and m/ = m T for 
the LFV contribution with coupling h^ T . We neglect the 
electron contribution as the coupling h^ e is more con- 
strained and because of the natural hierarchy assumed 
in formula (JsJ) - For the charged Higgs boson we have the 
same coupling for the scalar and pseudoscalar contribu- 
tions in the Lagrangian, therefore we give the sum of the 
two in one formula: 



2x 2 (x - 1) 



dx (13) 




where we neglected terms proportional to the neutrino 
mass. In order to give the bounds coming from the g — 2 
measurement we choose the sets of mass and mixing angle 
parameters of Table |. By calculating the contribution 



FIG. 2: Aa M in units of 10 -10 as a function of tan/3 using 
set 2 of parameters for the 2HDM-III type a (dashed line) 
and 2HDM-III type b (continuous line). The region above 
the horizontal line is excluded at 90 % CL by the muon g — 2 
data. 



TABLE I: The four sets of parameters used to obtain 
bounds on the LFV couplings. Set 1-3 are consistent 
with the relations between masses and mixing angles 
obtained at one-loop within the MSSM |27j in order to 
allow for a comparison. Note however that the 2HDM- 
III is not constrained by the symmetries imposed on 
MSSM in order to avoid tree-level LFV. Set 4 corre- 
sponds to a choice of parameters that is not allowed in 
MSSM. Masses are in GeV and the angle a in rad. 



Set 


m h 


m H 


■niA 


m H ± 


a 


tan/3 


(1) 


93 


134 


100 


127 


0.4 


5 


(2) 


127 


131 


129 


160 


-0.58 


45 


(3) 


128 


500 


496 


509 





50 


(4) 


125 


200 


200 


250 


0.2 


10 



III. COLLIDER EXPERIMENTS 

Bounds on other LFV couplings \j can be obtained 
from the processes B s — > [ifi, B — ► Y s/ u/i, e + e~ — > 
icu e u e , e + e™ — > ttcc, — * tc, B s — > (K)t/i and 
B s — > (K)tt |l|. However, the bounds obtained from 
these processes would involve two couplings as in the case 
of fx — > e'y. The flavor changing process t — > H°c has 
been extensively studied for the LHC (28) ; in the context 
of the 2HDM-I and II, top quark decays beyond the SM, 



t^ch (h = h°,H ,A ), 



(14) 



to a M from the Higgs sector we obtain limits on the LFV 
couplings of 2HDM-III type a and b — we use only the 
upper limits of (|ll]) to derive the muon g — 2 bounds 
on the LFV couplings. The results are in Table |fi|. In 



TABLE II: The 90% CL limits on the LFV couplings 
Ai-p, Ctmj i) t( i from the experimental measurement of 

<V 

Set 2HDM-III type a 2HDM-III type b 



(1) A TM < 31 (t? tm < 0.06) 

(2) A Tfl < 38 (j? tm < 0.07) 

(3) Ar/i < 123 (77 TM < 0.24) 

(4) A rM < 53 (r? TM < 0.10) 



A TM < 6.3 (£1-/4 < 0.012) 
A TM < 0.8 (£ T/4 < 0.002) 
Ar/i < 2.5 (£ rM < 0.005) 
A TM < 5.3 (£r„ < 0.010) 



Figure g we show the values of Aa^ given by the 2HDM- 
III using the set (2) of parameters of Table | with A M „ = 
10 as a function of tan (3. In model type a, Aa M is almost 
flat for tan/3 > 2, while in model type b it is a growing 
function of tan (3. The same is true for the other sets of 
parameters. In both models the Higgs sector contribution 
to Aa M is a growing function of the LFV couplings. 



were studied in ||2^] and it is shown that these processes 
could be accessible at the LHC and at the linear collider; 
the prospects for detecting the decay t — > cH at the e + e~ 
linear collider have also been investigated in [^0[ . 

The i/fi — v T flavor mixing observed in the atmospheric 
neutrino experiments ]3l|| would lead to the flavor violat- 
ing decays 







T± - 






■* XlTfJ,, 


h - 





SUSY can accommodate the observed flavor mixing Iq, 
|, and thus, the LFV processes ©, (|l|) and @ 
would arise in these models. A study conducted at the 
LHC showed that an upper bound of 0.6 x 10~ 6 on the 
— > /i ± 7 branching ratio can be achieved with an inte- 
grated luminosity of 30 fb _1 while theoretical estimates 
are at the level of 10~ 9 or less [|o[ Direct evidence of 
LFV in the slepton sector of SUSY would be inferred in 
the observation of the process ([l7]) which has also been 
studied for the LHC [03|. It was shown that in some 
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cases, the direct evidence would offer better sensitive 
than the — * /i ± 7 process. 

The decay h — > can be accommodated in the 

2HDM-III where no discrete symmetry suppresses the 
LFV couplings at tree level, and the partial decay width 
is parameterized by the LFV coupling A T/1 . The decay 
H° — > if IT (SM-like) will be used in the following as a 
comparison for the LFV decays. Its partial width is 



1 mj 
8^ ~v I 



(19) 




FIG. 3: Higgs boson production mechanism through gluon 
fusion. 



where we neglect small terms of the type m,/mjj — see 
the appendix for complete expressions. The partial width 
of the decay H° — > iflj (where I = e, n, r and i ^ j) is 
for 2HDM-III type a 



T(H° - if If) = m H 



and 



Ajj- miirij sin 2 (a — /?) 
^ 2 sin 2 /3 



Afj miirij sin 2 (a — /3) 
"8^r ^ 2 cos 2 /? 



(20) 



(21) 



for 2HDM-III type b. 

Hadron colliders may be sensitive to the processes h 



e ± /i T and h 



particularly at high luminosity, 



but these decays are not considered in the present study 
which is further motivated by a favorable interpretation 
of the atmospheric neutrino mixing experiments. It is 
shown in |35| that the muon collider would be sensitive 
to H° — > t^/i t . The non-observation of this process for 
rriH < 140 GeV at the muon collider in addition to the 
failure to detect the top quark decay t — > cH° at the 
LHC H would rule out the 2HDM-III ||. 

In this paper, we present the prospects for the detec- 
tion of the LFV decay A°/H° -► r ± /^ = F at the LHC and 
TeVatron. We shall consider the 2HDM-III and we shall 
parameterize the A°/H° — > r ± ^ T branching ratio (BR) 
by the LFV coupling parameter n Tfl 36 with respect to 
the SM-like decay H° — » t + t~ given in formula (19|): 



BR(A°/H° -» T fi) 



2m, 



BR SM (H° -» rr) 



(22) 

where the dependence on a, /3, the ratio of the total 
widths and A TM is absorbed into the LFV coupling pa- 
rameter K Tfl . For example, for the decay and the model 
considered in formula (EOj) we have: 



sin (a — (3) 



r " V2sin/3 V T% 
while from formula ( pl| ) we obtain: 



(23) 



sin(q - /?) /rf M 



(24) 



where T§, M 



is the total SM-like width and T?p b is the 



total width in model a, b respectively. Similar formulas 
can be written for the h° and ^4° Higgs bosons. For h° 
one has to replace sin(a — 0) in equations (^3|) and j2^ ) 
with cos(a — f3). For A one has to replace sin(a — 0) 
with 1. 

In Table |ll| we give examples of the correspondence 
between the parameterization in terms of A and the one 
in terms of k for set 1 and set 2 of Table 1. 



TABLE III: The correspondence between the parame- 
ters k and A using set 1 (set 2) of Table ffl for the LFV 



couplings of the Higgs bosons H u and A° 





A = 1 


A = 5 


A = 10 


type a 


k(A°) 


1.1 (1.2) 
3.3 (30.7) 


9.4 (6.2) 
16.6 (68) 


7.8 (12.2) 
7 (72.6) 


type b 


k(H») 
k(A°) 


1.1 (0.7) 
0.1 (0.001) 


5 (3.3) 
0.5 (0.006) 


7.6 (6.5) 
1 (0.01) 



We shall discuss the achievable bounds on k tm and X Tfl 
in the following sections. 



IV. SEARCH FOR A°/H° -»■ r/x 

We consider the production of the neutral Higgs bosons 
A and H° through gluon fusion, gg — > A° / H° (see 
Figure |), and the LFV decay A°/H° -> r ± ^^ (Fig- 
ure p. We restrict the present work to the low mass 
region, 120 < < 160 GeV, primarily because the 
SM decay H° s „ -► r+r - , hence A°/H° -> r+/i~ - 
see Equation (j2^) — becomes negligible j37j as the SM 
mode Hg M — » W + W~ opens up around 160 GeV as 
shown in Figure |^ where we assume k tm = 1 . We take 
as a reference the parameters of set 2 in Table [| for 
comparison with the MSSM case without loss of gen- 
erality. The events are generated in PYTHIA6.2 
with CTEQ5L []39"| parton distribution function parame- 
terization, and with the detector resolution and efficien- 
cies parameterization of ATLFAST [EJ from full detector 
simulations. 

We search for a final state where the r lepton de- 
cays to hadrons, r — > jeti/ T with a branching ratio of 



5 



H' 



FIG. 4: The Higgs decay through the lepton flavor violating 
coupling H°rfi. 
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FIG. 5: The Higgs boson decay branching ratios as a function 
of run- For the A°/H° — > r + (i~ +r~/i + channel, the coupling 
parameter k tm is taken to be one. 




FIG. 6: The reconstructed and the generated pr (top plot) 
andp z (bottom plot) of the r lepton. Equations ( prj| ) are used 
for the reconstructed quantities. 



of the figures referenced in the sections IV A| - |rV~C| are 
that of three years at low luminosity for on e ex periment 
at the LHC using the rates shown in Table [Fvl 



A. Hadronic r decay 



~ 65% or to an electron, r — > ev e v T {BR ~ 18%). 
The main backgrounds — in both final states — include 
the W + W~~ pair production, the Drell-Yan type process 
Z°(7*) — > t + t~. For the hadronic r decays, an addi- 
tional background comes from W /± +jets events where a 
jet is mis-identified as a r jet: 



PP(p) 



W ± Z° 

w+w~ 

ti 



Z°( 7 *) 



W ± +jets 



f^Vp, + jets. 



(25) 



The g g — > A°/H° cross sections are calculated using 
the program HIGLU (4^] . The signal cross-sections have 
been calculated at next-to-leading order (NLO) and next- 
next-to-leading order (NNLO) jid], [f2|. For the back- 
grounds, NLO estimates are available g§ |§, |§, 

ex- 
cept for W /± +jets where NLO calculations have been per- 
formed for a vector boson production with 2 jets at the 
TeVatron jf6|. We have therefore used the leading or- 
der (LO) estimates of the signal and background cross 
sections. 

Unless explicitly stated otherwise, the normalizations 



The event selection for the hadronic final state of the 
t lepton is carried as described below: 

(1) Search for one isolated muon (p T > 20 GeV, < 

2.5) to provide the experimental trigger, and one 
hadronic t jet (p T > 20 GeV, |7? r | < 2.5). We fur- 
ther require a jet veto and a b-jet veto — no other 
jet with pt > 20 GeV within |?7| < 2.5 — to reduce 
W +jets and ti — > b\x Jr v^bT~v T backgrounds. A r 
jet identification efficiency of 30% is assumed. 

(2) The 4-momentum of the r lepton is reconstructed 

from the r jet and the missing transverse momen- 
tum (using the prescription of Q) as follows: 



Pt = 
Pi = 
El = 



i? T ~j et + n misS 

Pt + Pt j 



(26) 



9l-j et 1 



->2 i 2 

p T +m T . 



p miss 
Pt 



The reconstructed momenta of the r lepton us- 
ing Equations ( p6[ ) are shown in Figure ^ together 
with the generated momenta. We demand that the 
hadronic r jet carries at least 60% of the r lepton 
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TABLE IV: The rates, a x BR(pb), for the signal gg -» A°/H° -> r+ /j~ + r~ /i+ , and the 
backgrounds at the LHC. The dominant backgrounds are Z — > rr and W +jets where W 7 ^ — > 
fl Ufx and a jet is mis-identified as a r jet. We assume the coupling parameter k tm = 1 and 
tan fi = 45 in the estimate of the signal rates. An additional background comes gg — » A /H° — > 
t + t~ with one r decaying to \i, r — ► [iv^Vt and the other r decays to hadrons. At tan/3 = 45, 
the scalar and the pseudo-scalar Higgs bosons are degenerate in mass for tha > 130 GeV, 
and the relative strengths of gg — > A — > r/i and — > if — > r/i are not important. For 
jtia = 120 GeV, the A and the H° bosons have comparable strengths. 



Process 




771a (^ev J 


rtiH (Oev J 




99 ^ A / H -> 




119.3 


128.4 


7.5 






1 9Q ^ 


lOU.l 


A r . 

-± . 






139.2 


140.2 


2.1 






149.1 


150.0 


0.8 






159.1 


160.0 


0.1 




TT 


119.3 


128.4 


99.5 






129.3 


130.1 


76.4 






139.2 


140.2 


54.3 






149.1 


150.0 


39.0 






159.1 


160.0 


28.5 




H ± V l j,T + T~ 




0.2 




pp^ IV+W" - 


> H+V^V-r 




1.67 




pp ^ tt — > f( 


&T f T & 




1.37 10 1 




pp^Z°( 7 *)^ 


T + T~ — > ^V^VrT^ 




1.39 10 4 




pp — > + jets 


—> fi Vfi + jets 




1.75 10 4 





energy and the cone AR — yj Ar/ 2 + A(f> 2 between 
the r jet axis and the t lepton direction be less 
than 0.2 rad: 



--jet 
Pt 

Pt 



> 0.6, 



(27) 



AR(p T * et ,p T ) < 0.2 rad. 



This cut reduces the background from + jets 
events by more than one order of magnitude while 
it costs only a modest ~ 40% rejection of signal 
events. 

(3) Using the tracker information in the off-line r iden- 

tification, we require that the r jet candidate con- 
tains a single charged track within AR < 0.3 rad 
around the jet axis. This cut would select one prong 
hadronic r decay events, and as shown in Figure |t], 
it reduces the + jets events by an additional 
factor of ten while costing only ~50% reduction in 
the signal reconstruction efficiencies. 

(4) The r lepton from the signal is ultra-relativistic, 

and as a result, the missing momentum from r — > 
(rjet)i/ is collinear with the r jet. Further, as a 
consequence of the two-body decay, the r jet and 
the /it track are back-to-back. We therefore require 
a large azimuthal opening angle between the \x and 
the r jet and a small opening angle between p™ iss 
and the r jet: 



' (pT'PT Jet ) 



> 2.75 rad, 



(28) 



fp miSf >T j6t 



< 0.6 rad. 



As can be seen from Figure ||, this cut reduces the 
signal by ~ 35% while the pp(p) — ► Z°(~f*) back- 
ground is further suppressed by ~51%. 

(5) The fi track is mono-energetic because of the two- 
body decay H° — > r ± /j t but the r jet in r — > 
(r jet)f would be somewhat softer. As a result, one 
would expect the momentum difference 



Ap T = p T -p T ' 



jet 



(29) 



to be positive for the signal. Indeed, as noted in |4q] 
and as shown in Figure ^, this quantity is very 
powerful in suppressing the pp{p) — > Z a (j*) back- 
ground further. 

(6) We now cut on the transverse momentum of the r 

reconstructed according to Equations (p6|). The 
distribution of this variable is shown in Figure [l^ 
where one sees that demanding p T > 50 GeV leads 
to at most 20% reduction in the signal — the r gets 
harder at higher vtla so the reduction in the signal 
due to this cut is highest at the lowest mass con- 
sidered, i.e., niA = 120 GeV — while the W^+jets 
and Z°{^*) — > t+t~ backgrounds are suppressed 
by additional factors of two and ten respectively. 

(7) The effective transverse mass of the r/i system 



rriT 



2p T p T J et [(l- cos 5<t>)} 



(30) 
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FIG. 7: The number of reconstructed charged tracks (arbi- 
trary normalization) within AR < 0.3 rad of the calorimeter 
jet axis. By requiring a single reconstructed charged track 
so as to select one prong r decays, the W^+jets background 
is further reduced by one order of magnitude while the sig- 
nal suffers ~ factor of two reduction consistent with the one 
prong hadronic r decay branching fraction. 



FIG. 9: The momentum imbalance Apr between the muon 
track and the r jet. In the signal, this quantity is expected 
to be positive as a result of the two-body kinematics from 
A°/H° —* and the subsequent decay r — * (r jet)^. This 

is indeed mostly the case as shown in the top left plot. There- 
fore, demanding Apr > suppresses the backgrounds further, 
particularly the Drell-Yan type process pp(p) — > Z°(^*) which 
is reduced by as much as 50% with this cut alone as can be 
seen from the bottom left plot. 
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FIG. 8: The azimuthal opening angle 8<f> between the muon 
track and the r jet (left plots), and between the p mlss vector 
and the r jet (right plots). The pp(p) — > Z°(j*) background 
is further reduced by a factor of two while the signal suffers 
only a 35% reduction. The dashed lines indicate the level of 
the cuts. 
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FIG. 10: The reconstructed transverse momentum of the r 
lepton. We require that this quantity be greater 50 GeV 
leading to additional suppression factors of two and ten in 
the dominant VF ± +jets and Z°(7*) — > t + t~ backgrounds 
whereas the signal is reduced by at most 20%. 
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FIG. 11: The reconstructed effective transverse mass of the 
t/x system. This distribution peaks at low values in the back- 
grounds while in signal the peak is closer to the actual Higgs 
mass. The dashed lines indicate the cut applied on this quan- 
tity. 



FIG. 12: The invariant mass Trirfj. distributions of the signal 
A°/H° — > t fi^ for several values of the Higgs boson mass 
and also of the backgrounds, after an integrated luminosity of 
30 fb _1 . The LFV coupling parameter = 1. The signal 
is reconstructed to within 5 GeV of the Higgs mass above the 
residual H /± +jets continuum. 



is reconstructed. In the signal, one would ex- 
pect this quantity to peak toward the Higgs mass 
whereas in the backgrounds, because the final state 
may contain several neutrinos, the ttit distribution 
would peak at low values as shown in Figure [ll]. 
We required that wit > 85 GeV. This cut sup- 
presses the Z° (7*) — > t + t~ background more than 
the other backgrounds. 

The efficiencies of the cuts discussed above are shown in 
Table [y] where one sees that the analysis steps described 
here is effective in reducing the two main backgrounds 
namely H/ ± +jets and Z° — > t + t~. The most effective 
cuts are the ones imposed for the identification of the r 
lepton — cuts (2) and (3) — and some kinematic cuts 
such as the momentum imbalance denned in cut (5). 

With the r 4-momcntum p T obtained in Equa- 
tions (p6|), the invariant mass of the Higgs boson is re- 
constructed, 



"TjU 



= (p T +p* 



(31) 



Distributions of m TM are shown in Figure [l^ for the signal 
and the backgrounds. We see in this figure that the 
signal is reconstructed within one GeV of the expected 
Higgs boson mass — except at wla — 120 GeV where 
the A and the H are not degenerate in mass and their 
summed signal peaks somewhere in the middle — while in 
the backgrounds the m Tfi distribution gives a continuum 
spectrum dominated by W /± +jets event. 



B. 



A°/H° r ± /xT versus A°/H° 



The H° — » t + t~ of the SM is not expected to yield 
a significant signal at the LHC due to a low signal rate 
and substantial backgrounds from various sources fl49| . 
In the MSSM, for a Higgs boson of the same mass, the 
A /H° — > t + t~ rates are significantly larger than the 
SM case. The A /H° — > t + t~ process has been studied 
extensively for the LHC, and it is demonstrated that such 
a signal can be observed with a significance exceeding 5a 
in a large area of the (m^, tan/3) plane |5(], 51 . 

The final state of both processes A /H° — > t + t~ and 
A°/H° — > t are very similar, namely an isolated 
/i, a hadronic r jet and missing energy. The observa- 
tion of these signals would rely on two crucial detector 
performance parameters, namely a very good p™ iss reso- 
lution and a very good r jet identification with excellent 
rejection of non r jets. The former performance param- 
eter is necessary for the reconstruction of the r/i invari- 
ant mass in A°/H° — > t ± /i =f (as demonstrated in the 
above analysis) and also for the rr invariant mass in 
A°/H° — > t + t~ |3(], |3lJ while the latter performance 
parameter allows for the suppression of various back- 
grounds containing fake r jets. We show in this sec- 
tion that the reconstruction procedures presented in this 
paper for A°/H° — > r/i and described in |5(], |5lJ for 
A /H° — > tt allow for the identification of each of these 
processes although their final states are similar. 
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TABLE V: The efficiencies (in percent) of the cuts used in the current analysis. The first three cuts are effective 
in reducing the dominant W +jets events while the other cuts suppress the rest of the backgrounds efficiently. 







J± 1 ll 




T [1 
















120 


l-OU 


140 


J-<JU 


i fin npv 


tt 


VV Zv 


w + w~ 

VV vv 


a \1 ) 


VV T^JCLD 


(i) 


16.3 


17.0 


16.9 


17.2 


20.9 


3.0 10 _i 


7.2 


16.3 


0.2 


0.5 


(2) 


9.1 


9.6 


9.7 


9.9 


12.1 


1.5 10" 2 


0.50 


1.3 


0.09 


2.2 10~ 2 


(3) 


4.4 


4.8 


4.6 


4.9 


5.9 


0.7 10~ 2 


0.25 


0.65 


0.04 


2.2 10~ 2 


(4) 


2.7 


3.0 


2.9 


3.1 


3.8 


2.7 10" 3 


0.10 


0.40 


0.02 


1.9 10" 3 


(5) 


2.4 


2.7 


2.6 


2.8 


3.5 


2.4 10" 3 


0.10 


0.40 


0.01 


1.1 10~ 3 


(6) 


1.7 


2.1 


2.2 


2.5 


3.2 


2.2 10~ 3 


0.05 


0.20 


2.4 10" 4 


5.3 10" 4 


(7) 


1.2 


2.0 


2.2 


2.5 


3.1 


2.1 10" 3 


0.05 


0.20 


1.6 10" 4 


4.3 10~ 4 



2. Optimization for A°/H° 



itu = 1 30 GeV - 
^=1.0 : 
ton/?=45 - 

m 4 = 1 30 GeV - 
ton/? = 45 




FIG. 13: The reconstructed m rfl invariant mass for A°/H° — > 
r/x and A°/H° -» r+r" m A = 130 GeV and k tm = 1 (tan/3 = 
45, a = —0.58 rad), i.e. set 2 of Table | using the analysis 
procedure presented above. The existence of the A°/H° — > 
t + t~ signal would constitute an additional background for 
the A°/H° — > r^fj,* process. 



It is also important to show that the analysis technique 
optimized for the search for the A Q /H° — ► tt signal is ca- 
pable of separating the tt final state from the t/j, events. 
We have therefore examined A°/H° — > t/j, events ac- 
cording to the A /H° — ► tt analysis technique which we 
recall succinctly as follows [[30| |l) : 

(a) One isolated /i with px > 24 GeV and |?7| < 2.5, one 

hadronic r jet with E 3 T et > 40 and \r)\ > 2.5 and 
b-jet veto. 

(b) E™ iss > 18 GeV. 

(c) The transverse mass mr(lepton-.E™ ss )< 25 GeV. 

(d) 1.8 < A(f> < 2.9 rad or 3.4 < A0 < 4.9 rad, where 

A(j) is the azimuthal opening angle between the t 
jet and the isolated /i. This cut is needed for the re- 
construction of the tt invariant mass m TT . Indeed, 
the invariant mass m TT of the pair of t leptons pro- 
duced in the process 



A°/H° 



jet VrflV^Vr 



can be reconstructed assuming that m T = 0, that 
the t detected products (in this case the r jet and 
the \i) are not back-to-back, and also that the di- 
rection of the neutrino system from each r decay 
coincides with that of the detected product: 



1. Optimization for A°/H° — > Tji 



We generated A°/H° — > tt events and analyzed 
them acco rding the analysis procedure described in sec- 
Thc relative efficiencies of the cuts described 
Tfi and A°/H° 



tion 



[VA 



in section gVg for A°/H° -> t/j, and A {) /H° -► tt final 
states are shown in Table |vi]. From Figure [l3[ we see 
that at the same Higgs boson mass, the reconstructed t/i 
invariant mass for the A / H° — > t + t~ events peaks at 
lower values. 



n TT = y/2(E 1 + E yl ){E 2 + E u2 )(l - waff). (32) 

Ei and E 2 are the visible energies from the r de- 
cays, 9 is the angle between the directions of the de- 
tected products, and E v \ and E u2 are the energies 
of the two neutrino systems, obtained by solving 
the system of equations 



[Euxui] x[y) + [E 



where u\ and u± are the directions of the detected 
products, and p™ lss and p" llss the components of 
the E™ 1SS vector. The above system of equations 
can be solved if the determinant, which is propor- 
tional to sin A<fi, is not zero. Further details of 
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TABLE VI: The relative efficiencies (in percent) of the cuts used in the 
current analysis for A /H° — > rr events, to be compared to Table |v] where 
the efficiencies for A /H° — > t\i are shown. 



Cut 




A /H 




TT 






120 


130 


140 


150 


160 GeV 


(1) 


2.0 


4.2 


2.1 


1.95 


1.77 


(2) 


0.6 


1.1 


0.6 


0.52 


0.47 


(3) 


0.3 


0.6 


0.3 


0.26 


0.23 


(4) 


0.1 


0.24 


0.12 


0.11 


0.10 


(5) 


0.09 


0.18 


0.09 


0.08 


0.08 


(6) 


0.01 


0.04 


0.02 


0.03 


0.03 


(7) 


0.8 10~ 2 


0.02 


0.02 


0.02 


0.03 



TABLE VII: The relative efficiencies (in percent) of the cuts used in the 
search for A /H° — * tt |jo[ [jlf and re-stated briefly in the text (raj = 
130 GeV). 



Cut 


A U /H U -> r± M + 


A [, /H [ > - 


-> TT 


(a) 


15.1 


3.1 




(b) 


5.3 


1.9 




(c) 


0.2 


1.5 




(d) 


0.02 


0.3 
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tan/3 = 45 



■ A"/H° -> TfM 
= 1 30 GeV 
v\„« 1 .0 
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FIG. 14: The reconstructed m TT invariant mass for A°/H" -> 
rr and A°/H° — » t/x, for = 130 GeV and « TM = 1 
(tan/3 = 45, a = —0.58 rad), i.e. set 2 of Table | using the 
analysis procedure presented in J5(], ^fj . The existence of the 
A /H° —*Tfi signal would constitute a negligible background 
for the A°/H° — > tt process. 



the m TT reconstruction are well documented else- 
where |g0, 51 . 



The relative efficiencies of the cuts (a)-(d) for the t/x 
and tt final states are shown in Table VII. Figure [l4| 



shows the reconstructed m TT invariant mass distribution 
for both final states. The t\i events would contribute a 
negligible background under the tt signal. 

The reconstruction procedure for A°/H° — > t + t~ de- 
scribed in |30| [5l[] and the analysis steps presented above 
for A 1 H° -^T^fi T , would allow for the separation of 
both signals, with each contributing a small residual 
background under peak of the other as shown in Fig- 
ures O and nAl 



C. Leptonic r decay 

Thus far, we have considered the hadronic final state 
of the t lepton, and the major irreducible background 
comes from W+jets events where a jet is mis-identified 
as a hadronic r jet. Indeed, the residual SM background 
shown in Figure [l2] is dominated by VF+jets events whose 
rate is several orders of magnitude higher than the signal 
rates as shown in Table |y|. In this section, we examine 
the leptonic decay of the t, namely t — * ev e v T . Although 
the branching fraction of t — > ev e v T is only ~ 18% com- 
pared to 65% for t — > (jet)v Tl the identification of the 
electron is easier with an efficiency of 90% whereas the 
t jet identification efficiency is much lower: in the above 
analysis, we assume a r jet identification efficiency of 
30%, corresponding to a jet rejection factor of ~ 400 — 
see |5(| for details. Furthermore, the leptonic decay of 
the t will not be sensitive to the VF+jets background. 
We search for a signal final state containing two isolated 
leptons, one electron and the other a /i with no hadronic 
activity. The major SM backgrounds in this case are 
shown in Table |v| — processes listed in (^5|) — except 
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TABLE VIII: The rates, a x BR(pb), for the signal gg -> A°/H° -» r+ > _ + t~/x+, and the 
backgrounds at the TeVatron. We assume k tm = 1 and tan /3 = 45. 



Process 


rriA (GeV) 


m H (GeV) 


a x BR(pb) 


a n rrm x x 

gg^, A l >/H l > ^t + h + t ^ 


119.3 


128.4 


1.41 10 _1 




129.3 


130.1 


0.79 10" 1 




139.2 


140.2 


0.33 10" 1 




149.1 


150.0 


0.11 10 _1 




159.1 


160.0 


0.15 10~ 2 


gg - ^/ff u - rr 


119.3 


128.4 


3.90 




129.3 


130.1 


2.84 




139.2 


140.2 


1.79 




149.1 


150.0 


1.16 




159.1 


160.0 


0.75 


pp — ► ^ u (7*) — ► T + r" — > n + VpV T T~ 




3.24 10 3 




pp — » W + jets — > /i*^ + jets 




3.21 10 3 





A°/H° — > t/j. (t — > ewf) 
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FIG. 15: The reconstructed invariant mass m TM in the lep- 
tonic decay of the r (— » ew) for the signal (raj = 140 GeV, 
tan/3 = 45, k = 1), the SM backgrounds and the A°/H° -» rr 
background. 



FIG. 16: The discovery reach at the TeVatron in the combined 
r — > jet v T and r — > ev^v-r channels. The signal would yield 
a 5a significance for 0.87 < k tm < 2.0 and for a Higgs boson 
mass 120 < < 150 GeV. The luminosity needed for a 95% 
confidence level exclusion is shown in the bottom plot. 



for the IF+jets background, in addition to: 

tt — > [iv^bevj), 
WW -> iiVp,ev e . 



(33) 



The re constr uction of the signal is exactly as described in 
section [V A , except for the cuts (2) and (3) which were 
implemented for the suppression of the VK+jets events 
and for the selection of the one prong hadronic t de- 
cays. These cuts are no longer necessary and are not 
used in the search for the leptonic decay of the r. Fig- 
ure |l5| shows the reconstructed t\i invariant mass for the 
signal A° /H° — > r/x, the SM backgrounds and for the 
A°/H° — > tt background with one r decaying to lep- 



tons: t — > evv. In this channel too, the signal can be 
observed with significances exceeding ha depending on 
the LFV coupling parameter k tm . 



D. Prospects at the TeVatron 



Table |VIII shows the estimated signal and background 
rates at the TeVatron where we propose to search for 
A /H° — > r ± /i =F with the neutral Higgs bosons of the 
2HDM produced through gluon fusion: gg — > A° /H°. 
The signal-to-background ratios and the signal signifi- 



12 




FIG. 17: The achievable lower bounds on the LHV coupling 
A Tfl at the TeVatron (CDF + D0) obtained from the summed 
signal in the hadronic and leptonic decays of the r lepton. To 
obtain these bounds, we use the values of k t ^ at 5<r shown 
in Figure [l6| — 20 fb" 1 per experiment. The current bounds 
on X Tfl obtained from the muon g — 2 data are also shown. 
For the muon g — 2 data, the curves at higher A Tfl values 
correspond to the 2HDM-III type a and the lower curves to 
the 2HDM-III type b. This trend is the same for H° -> T[i 
(bottom plot) but it is reversed for A — » t/i (top plot). 



FIG. 18: The reconstructed invariant mass m TM , after cut (7), 
of the signal plus the backgrounds in the hadronic r decay 
channel for rriA = 120, 130, 140 and 150 GeV, and for an 
integrated luminosity of 30 fb -1 at the LHC. For the assumed 
value of the LFV coupling parameter (k tm = 1), the signal 
can be observed with a significance exceeding 5<r up to tua = 
150 GeV. 



their different LFV Yukawa couplings — see sections 
III and the appendix. 



cances calculated within ±2cr of the reconstructed Higgs 
mass peak, for an integrated luminosity of 20 fb -1 per 



experiment, are shown in Table IX for = 1 and 



tan/3 = 45. At the TeVatron, a significant signal (> 5cr) 
can be detected for Higgs boson masses around 120 GeV 
and high tan/3 (~ 45), assuming n Tfl ~ 1. We show 
in Figure H the discovery reach at the TeVatron and 
the luminosity required for a 95% confidence level ex- 
clusion for large tan/3. For low tan/3 values ( < 10), 
the signal production rate decreases by more than an 
order of magnitude compared to the case shown in Ta- 
ble VIII so that the detection of this process at the TeVa- 



tron would require very large values of the LFV coupling 
At,,. However, one would expect the LFV couplings 



0(1) 



see Equations (E3|) and (24) 



E. Prospects at the LHC 

The signal and background rates at the LHC are shown 
In Figures [11] and [l9[ we show the re- 



in Table [V 



constructed m Ttl invariant mass for several values of the 
Higgs mass, and for an integrated luminosity of 30 fb -1 , 
and for the LFV coupling parameter K£„ = 1. 

The signal-to-background ratios and the signal sig- 
nificances are calculated with the events reconstructed 
within ±2cr of the reconstructed Higgs mass. As shown 
m Table [x|, a significant signal can be observed at the 
LHC for Higgs masses in the range 120 to 150 GeV for the 
LFV coupling parameter n TfJ _ ~ 0(1). Around 160 GeV, 
as the Hg M — > W + W~ channel opens up, the rate for 



Therefore, at the TeVatron, this channel would be viable A°/H° -> decreases so drastically that the obser 

only in the event of a large tan/3 value and for k tm - 1 vation f a significant signal would be possible only in 
— see Table [H| for the correspondence between K Tf2 and the event of n Tfl > 1. 
X Tfl . Figure ]l7| shows the corresponding expected lim- 
its on X Tfl at the TeVatron: the reach in X Tfl would be 
extended, at large tan /3, beyond that obtained from the 
muon g — 2 experiment, for A — > t/i in the 2HDM-III 
type a. 

As shown in Figure [l?], the bounds on A TM would be 
different for the Higgs bosons ^4°, H° and h° because of 



The constraints on this LFV coupling A TM from low 
energy experiments are rather weak — see the discussion 
in section |h| on low energy bounds. From Equation (|2^), 
the signal rate scales like k tm and we show in Figure |2fj| 
the value of k tm at which the signal yield a 5a signif- 
icance around the Higgs boson mass peak. The LFV 
coupling 0.18 < K Tfl < 1.0 can be reached at the LHC, 
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TABLE IX: The expected signal-to-background ratios and signal significances (r — > 
jetv/r — ► evv) for two experiments at the TeVatron, assuming fi^ M = 1 and 5% 
systematic uncertainty on the background shape and normalization. 



m A (GeV) -> 


120 


130 


140 


150 


Signal (S) 


10/29 


7/19 


3/13 


1/5 


Backgrounds (B) 


4/42 


4/44 


3/51 


2/62 


S/B 


2.4/0.7 


1.8/0.4 


1.0/0.3 


0.5/0.1 


s/Vb 


5.0/4.3 


3.5/2.7 


1.7/1.7 


1.2/0.6 


Combined S/V~B 


6.6 


4.4 


2.4 


1.3 



TABLE X: The signal-to-background ratios and signal significances calculated within ±2<r of the 
reconstructed Higgs mass < ra A > for r — > jet v/t — » evv — one experiment at the LHC — with an 
integrated luminosity of 30 fb^ 1 , assuming = 1 and 5% systematic uncertainty from the residual 
background shape and normalization. 



m A (GeV) -> 


120 


130 


140 


150 


160 


<m A > (GeV) 


124.6/125.2 


130.0/130.7 


139.9/140.6 


149.7/150.0 


159.4/159.8 


a (GeV) 


7.5/7.3 


7.0/7.0 


7.6/8.2 


8.1/9.1 


8.4/10.4 


Signal (S) 


943/1142 


687/816 


349/624 


144/279 


23/57 


Backgrounds (B) 


360/134 


397/140 


376/163 


296/198 


223/226 


S/B 


2.6/8.5 


1.7/5.8 


0.9/3.8 


0.5/1.4 


0.1/0.3 


s/Vb 


36.1/85.4 


24.4/59.4 


12.9/41.2 


6.3/16.2 


1.2/3.0 


Combined S/VB 


92.7 


64.2 


43.2 


17.4 


3.2 



A°/H° -» t jx (t -» evv) 




100 125 150 175 200 100 125 150 175 




100 125 150 175 200 100 125 150 175 200 



m T „ (GeV) 

FIG. 19: The same as Figure [l^ but with the leptonic decay 
of the r (— > ev e v T ). 



combining ATLAS and CMS data for Higgs boson masses 
120 <m A < 160 GeV. Figure |o| also shows in the bot- 
tom plot, the luminosity needed at the LHC to achieve 
a 2(7 (95% CL) exclusion. At the LHC, assuming the 
LFV coupling parameter K TfJ _ ~ 0(1), few years of low 
luminosity data would be enough to exclude this model 
in mass range 120 < rriA < 150 GeV and at low tan/3. 



For high tan/3 values, a 95% CL exclusion can be es- 
tablished in one year of data taking or less for the mass 
range considered. In Figure pi], we show the expected 
bounds on A^u at the LHC using the values of k tm at 5<r 
from Figure |20| and we see that the reach in X Tfl would 
be extended beyond the muon g — 2 limits. 



V. CONCLUSIONS 

In models with several Higgs doublets, FCNC and LFV 
couplings exist at tree level because the diagonalization 
of the up- type and the down-type mass matrices does not 
ensure the diagonalization of the Higgs-fermion coupling 
matrices. In the 2HDM-I and II, a discrete symmetry 
suppresses FCNC and LFV couplings at tree level by re- 
stricting fermions of a given electric charge to couple to 
at most one Higgs doublet. In the 2HDM-III, no discrete 
symmetry is invoked and the flavor changing couplings 
are parameterized in terms of the fermion mass hierar- 
chy to be in agreement with the severe experimental con- 
straints on FCNC and LFV couplings with the first gen- 
eration index. The arbitrariness of the FCNC and LFV 
couplings of the second and the third generations can be 
constrained in low energy and collider experiments. The 
deviation of the measured muon anomalous magnetic mo- 
ment from the SM prediction offers weak bounds on the 
LFV coupling parameter A TM . 

We have investigated the achievable bound on n Tfl and 
\ T fi at hadron colliders by studying the gg — > A°/H° — > 
t fj^ signal observability. 

Considering the hadronic decay of the r lepton, the 
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FIG. 20: The 5a discovery reach in the (niA, kw-m) plane, using 
the combined r — > jet v T and r — > ev E v T signals, for ATLAS + 
CMS (top plot). The A°/H° -> r=V T signal would yield a 5cr 
significance for 0.18 < k tm < 1.0 and for a Higgs boson mass 
120 < mA ^ 160 GeV. The bottom plot shows the luminosity 
needed for a 95% confidence level exclusion as a function of 
vtia for low and high tan/? assuming k t/j = 1. 



FIG. 21: The achievable bounds on the LHV coupling A rfJ at 
the LHC (ATLAS + CMS) from the combined hadronic and 
leptonic r decay channels — 100 fb _1 per experiment. For low 
or high tan (3, the lower bounds on A TM at the LHC would be 
extended beyond that of the muon g — 2 experiment. For the 
muon g — 2 data, the curves at higher A Tft values correspond 
to the 2HDM-III type a and the lower curves to the 2HDM-III 
type b. This trend is the same for H° — > r/i (bottom plots) 
but it is reversed for A" — > t/i (top plots). 



main backgrounds of this process are Z (7) — > t + t~ 
and W^+jets events where — > // ^ and a jet is 
mis-identified as a r jet. We search for an isolated /z and 
one r jet, and we applied a jet veto and a b-jet veto to 
reject multi-jet final states from ti and VF ± +jets. Further 
reduction of the backgrounds is achieved by exploiting 
the differences in the event topology of the signal and 
the various backgrounds. Although the background rates 
are several orders of magnitude higher than the signal 
rate, three main detector performance parameters have 
been crucial in extracting a significant signal: a good 
r jet identification and rejection against non r jets, the 
tracking capability for the identification of the charged 
tracks in one prong hadronic r decays, and the missing 
momentum resolution. 

We also investigated the leptonic decay of the r (— > 
evv) where the VF+jets event do not contribute a signif- 
icant background. In this case, we require a final state 
containing one isolated fi, one isolated electron, and we 
use jet veto and b-jet veto to suppress the ti background. 
The leptonic decay of the r gives a better sensitivity. 
The signal significances are estimated based on expected 
events in both the hadronic and the leptonic r decay 
channels. 

The analysis steps described above reconstruct the t/i 
invariant mass to within ~ 1 GeV of the Higgs boson 
mass (except at itia = 120 GeV where the A° and the H° 
bosons are not degenerate and the summed signal peaks 



somewhere in the middle as a result), and also differenti- 
ate the A°/H Q -> r+r~ events from the A°/H° -> r ± /i =F 
signal. 

With an integrated luminosity of 20 fb _1 at the 
TeVatron, a signal for 0.87 < n Tfl < 2.0 could be de- 
tected with a significance of 5er for Higgs boson masses 
120 < tua ^$ 150 GeV, corresponding to a reach in X Tfl 
of the order 0.03/1.0 for the 2HDM-III type a/type b. In 
case the signal is not observed and assuming K Tfl ~ 0(1), 
a 95% CL exclusion can be set with < 14 fb _1 of data 
for 120 < m A < 140 GeV. 

The sensitivity will be improved at the LHC where 
k t/j ~ 0.18 could be reached with 100 fb _1 — this would 
correspond to a lower bound in X Tfl ~ 0.01/0.1 for the 
2HDM-III type a/type b. A 95% CL exclusion could be 
set after just a few years of running at low luminosity if 
the signal is not observed. 

At the LHC, the reach in the LFV coupling \ Tf2 would 
be extended beyond that expected at the TeVatron and 
also beyond that obtained from the muon g — 2 data: a 
factor of 10 to 100 better depending on tan/3 and a. 
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Appendix 

In this appendix we give the Lagrangian of the lep- 
ton flavor conserving and violating Yukawa couplings of 
the 2HDM type III using the results of 
dition of the charged Higgs part. The 
Lagrangian reads: 



2 with the ad- 



cptonic Yukawa 



- C = r,i$L<Pil% + ^MlV^R + h.c. 
where (pi t 2 are the two Higgs doublets 



(34) 



with vacuum expectation values 



(^2)0 = 



\/2 \ V 2< 



ill 



(36) 



where in the following we set 9 = and therefore we 
consider a CP conserving Higgs sector. The parameters 
rjij and £y are non diagonal 3x3 matrices and i, j are 
family indices. The neutral and charged mass eigenstates 



' cos/3 


— sin /3 \ 


y sin (3 


cos/3 J 


I cos/3 


— sin /3 ^ 


I sin/3 
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/ cos a 


— sin a 


I sin a 


COS Ot , 
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G% 
.4° 

H° 
h° 



V2 



30? 
30° 



(37) 



where 3?0 and 30 are the real and imaginary parts of the 
complex scalar fields 0, tan/3 = v 2 /vi, a is the CP-even 
neutral Higgs sector mixing ang le, G° z and G± arc the 
would-be Goldstone bosons of Z and W vector bosons, 
and H^, A , H°, h° are the physical Higgs bosons of 
the 2HDM. The Lagrangian |34|) in terms of the mass 
eigenstates is obtained by a unitary transformation 



Il,r — Vl,r 



(38) 



and one can write the diagonal mass matrix for the three 
leptons 



M dia g = Vt ( vi 



V-2 



(35) and cither solve for £ (rotation of type a) 



£ = ^LvlMf^Vn - % 
v 2 v 2 



or for 77 (rotation of type b) 



V2 T/ f »,rdiag T/ , V 2 

V = v l M i Vb. £• 

Vl Vl 



(39) 



(40) 



(41) 



In terms of 77 the leptonic Lagrangian reads (type a): 



C a = ^[J4cosa/i +sinai/ )-z^^^ 5 ^A°-^^n i (l+ 75 )/ 4 i/+ (42) 

wsm/3 v y/2v 

1 - i - 1 

krujlj [cos(a - P)h° + sin(a - f3)H°] + , J iVijlsljA + —— ^^(l + j 5 )ljH + + h.c. 



' y/2 sin j3 " ' v " 1 ' V2 sin/3 * ' y,0 J 2sin/3 

while in terms of £ the leptonic Lagrangian reads (type b): 

~h m i t , , . , n rT n> mitan/3 r , , n m^tan/3 . . , „, 

C b = —kk (sina h° -cos a H°) + i— -l^kA + -= f nA\ + -y 5 )kH + (43) 

v cos p v v 2v 

1 - i - 1 

k&jlj [cos(a - 0)h° + sin(a - p)H°] - -= -h&j^ljA - 3^(1 + "fo)ljH + + h.c. 



V2cos/3" JL v v^cos/3 J J 2cos/3 

I 

where n is the neutrino field, v = (\/2 Gf) 1 ^ 2 = to v\ and v 2 by 
246 GcV is the SM vacuum expectation value, related 



V = \Jvl+vl 



(44) 
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Note that the Lagrangian (|4^) corresponds in the lepton 
flavor conserving part to 2HDM-I, while the Lagrangian 
(||) to 2HDM-II. 

The couplings for lepton flavor conserving and violat- 
ing Yukawa interactions hij can be read directly from the 
Lagrangian (^) and (f43[). As an example the lepton fla- 
vor conserving charged Higgs coupling squared from J43| ) 
(which is the same as in 2HDM-II) is 



m? tan 2 (3 



Gf to 2 tan 2 (3 



2v 2 



V2 



(45) 



in agreement with the erratum in |T| (see the discussion 
there for a comment on other results in the literature). 



We give in the following the complete expressions for 
the widths used in the analysis. For tree- level widths we 
do not give here loop contributions and threshold effects, 
but those effects are taken into account in the numerical 
calculation of the branching ratios. The decays of a CP- 
even/odd neutral Higgs boson to a pair of fermions are: 



r(.ff - ipj) = 



m H —^- — 2-^r(a,/3) 



/ _ {mi+m 3 ) 2 \ 3/2 / _ (m,~m 3 ) 2 \ 1/2 

V m H J \ m H J 



T(A°^l+lJ) = m A - 



c*ij rrnmj 2 



(a, 0) 1 



/ \o \ 1/2 / / \9\ 3/2 

(rrii + wij) \ f [mi — rrij) \ 



(46) 
(47) 



where iV c = 3 for quarks and N c = 1 for leptons. For a 
flavor conserving decay Ajj = 1 and = rrij. Oja, (3) is 
a function of the mixing parameters, given in Table XL 



The Higgs couplings to gauge bosons follow from gauge 



invariance and are therefore model independent. There 
are no tree-level couplings of vector boson pairs to the 
charged Higgs H ± and to the CP-odd neutral Higgs bo- 
son A. For the neutral CP-even sector: 



T(h° -> W+W-) 



T(h° -» ZZ) 



sin 2 (/3 



sin 2 (/3 - a) 



4771^771^ 



\2m" w ) 



I2m% 



1-4- 



1/2 



1-4- 



1/2 



(48) 
(49) 



and the corresponding expressions for H° can be ob- 
tained replacing sin 2 {(3 — a) with cos 2 {[3 — a). The loop- 
induced decays to gg and 77 can be obtained from chap- 
ter 2 and appendix C of the first reference in (l) for the 
MSSM. For a generic neutral Higgs boson cf> they are 
given by: 



r(</>°-77) = 



2567^ m * I I > 



(50) 
(51) 



where the sum over the index i is limited to quarks for 
99 



J h q = C*F 1/2 ( Tq ) 
while it runs over fermions, W, H for 77: 
If = N c e 2 CfF 1/2 (r f ) 



(52) 



(53) 



Mr 



4 ± = C% ± F (t h± )- 



<'W 



(54) 
(55) 



where Tj = Amf/m^, N c = 3 for quarks, 7V C = 1 for 
leptons, e is the electric charge in units of the charge of 
the electron, the functions F are given by 

F - r [I-t^t)] (56) 
F 1/2 = -2t[6+(1-St) 9 {t)} (57) 
Fx = 2 + 3t + 3t(2-t) 5 (t) (58) 

where 6 = 1 for h°, H°, and 6 = for A. The function 

g(ar) is 



9(x) = 



asin 2 (-y/ 1 / x) 



1+ Vl-g: 



X > 1 
X < 1 



(59) 



The coefficients C 2 are given in Table XII and the cou- 
plings g(h°H+H-), g{H n H + H-) of Table [XIl| can be 
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TABLE XI: The mixing angles for the neutral Higgs bosons decays to 
fcrmions; u refers to up-type quarks, d to down-type quarks and leptons. 







MSSM 


2HDM-III type a 


2HDM-III type b 


8(h u - 


Tuuj 


P/~iQ fx /ti.Tl ri 

LUB Ll / Olll LJ 


LUD L-l / Oil! LJ 


tiin /T/ /r*nti r? 

13111 Lt / LUB LJ 


0(h° - 


* dd) 


sin q / cos f3 


cos a / sin fi 


sin a / cos (3 


e(h°- 


* rn) 





cos(a-/3)/(^2sin /3) 


cos(a-/3)/(V2cos/3) 


8{H° - 


-* mm) 


sin a / sin /3 


sin a 1 sin /3 


cos a / cos j3 


0(H° - 


-» dd) 


cos a/cos /3 


sin a 1 sin /3 


cos a / cos f3 


9{H° - 


-f Tfj) 





sin(a - /3)/(\/2sin /3) 


sin(a - /3)/(\/2cos /?) 


e(A° - 


-» mm) 


cot/3 


cot /3 


tan /? 


0(A° - 


-f dd) 


tan /3 


cot j3 


tan (3 


e(A° - 







1/(^2 sin /3) 


1/{V2 cos /3) 



TABLE XII: The coefficients Cf ; m refers to up-type quarks, d to down-type 
quarks and leptons. 
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